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THREE-DIMENSIONAL ENHANCEMENT PROCESSING FOR A 
TELEVISION BROADCASTING SIGNALS 

FIELD OF THE INVENTION 

This invention relates to image processing technologies generally and 
particularly to video decoding systems. 
BACKGROUND OF THE INVENTION 

In order to conform to the National Television Standards Committee 
(hereinafter NTSC)/Phase Alternating Line (hereinafter PAL) standards for color 
television, a NTSC/PAL encoder interleaves the luminance and chrominance 
components in an intricate manner to conserve bandwidth and ensure compatibility 
with monochrome televisions. However, as the NTSC/PAL encoder processes 
higher resolution inputs because of the usage of higher resolution cameras, 
computer-based graphics animations, etc., proper decoding of such inputs is 
becoming increasingly difficult for a traditional NTSC/PAL decoder. 

Figures 1 illustrates prior art digital decoder 100 for a NTSC/PAL television 
receiver. Digital decoder 100 operates on composite video signals that conform to 
NTSC/PAL standards in three main stages. Specifically, after analog-to-digital 
(hereinafter A/D) converter 102 generates digitized composite signal 104 based on 
the incoming signals, Y/C separator 106 separates out luminance (Y) and 
chrominance (C) from this digitized composite signal 104. Because display 



subsystem 110 typically utilizes cathode-ray tube (hereinafter CRT) technology that 
uses red, green and blue phosphors to create the desired color, signal adjuster 108 
operates further on the newly obtained Y and C signals to generate R (red), G (green) 
and B (blue) signals. Some of the functions that signal adjuster IQ^^performs are, 
but not limited to, chrominance demodulation, brightness, contrast, saturation and 
hue adjustment, display enhancement processing, color space conversion, pixel 
formatting, etc. 

Figures 2(a) and 2(b) illustrate prior art Y/C separator 200 and prior art Y/C 
separator 220, respectively, for most of the existing NTSC televisions. In particular, 
Y/C separator 200 is an ordinary two-line comb filter that operates on digitized 
composite signal 104 as shown in Figure 1. Y/C separator 200 employs first line 
delay element 202, second line delay element 204 and adder 206 to generate a 
double-amplitude output composite video signal 208 (since the sub-carriers for 
digitized composite video 104 are in phase). Because of a 180*^ phase difference 
between output composite video signal 208 and delayed composite video signal 212, 
subtracting the two signals cancels most of the luminance and leaves double- 
amplitude chrominance. Bandpass filter 216 further eliminates signals outside of a 
predefined frequency range for chrominance information to yield signal C. 
Adder/subtractor 218 then subtracts signal C from delayed composite video signal 
212 to generate signal Y. 




Y/C separator 200 however has trouble with diagonal lines and vertical color 
changes. With diagonal lines, Y/C separator 200' s C signal as shown in Figure 2(a) ^ 
includes the difference between adjacent luminance values, which may be 
interpreted as chrominance information. Such interpretation results in cross-color — 
artifacts. As for vertical color transitions, the so-called "dot crawl" artifact surfaces 
due to imperfect removal of color information from Y/C separator 200' s Y signal as 
shown in Figure 2(a). 

Y/C separator 220 as illustrated in Figure 2(b) is an improved solution over 
Y/C separator 200. Particularly, Y/C separator 220 includes vertical correlation 
detector 222, which determines the amount of line-to-line correlation. If line 1 
(hereinafter LI) and line 2 (hereinafter L2) are highly correlated, vertical correlation 
detector 222 causes multiplexer (hereinafter Mux) 224 to select L2-L1 as its output. 
If L2 and line 3 (hereinafter L3) are highly correlated, the output of Mux 224 is then 
L2-L3. Although Y/C separator 220 may address the aforementioned cross-coloT^ 
and dot crawl artifacts, it may have problems with real-world video. More 
specifically, it may not have sufficiently correlated lines to adequately separate the 
luminance and chrominance signals as occurs with high resolution cameras and 
computer graphics animation. So, the more detail that is present in digitized 
composite video 104, the greater the artifacts would be introduced by Y/C separator 
220. 



As for many of the existing PAL televisions, they utilize Y/C separators that 
are similar to their NTSC counterparts as shown in Figures 2(a) and 2(b). One 
difference is that a PAL Y/C separator also has a PAL modifier that provides a 90^ 
phase shift and removal of the PAL switch inversion. However, the PAL Y/C ^ 
separator suffers from Hanover bars that result from a real and complementary hue 
error between pairs of adjacent lines and also cross-color as mentioned above. 

As has been demonstrated, the decoding technologies that exist in many of 
the current NTSC/PAL televisions are unable to keep pace with the ever increasingly 
dynamic, complex and detailed video sources. Thus, an apparatus and method is 
needed to provide a cost-effective, flexible and highly scaleable video decoding 
solution that not only could acconmiodate the discussed shortcomings of the current 
NTSC/PAL television decoders, but also could further improve display quality of 
NTSC/PAL composite signal on other types of display equipment. 



BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention is illustrated by way of example and is not limited by 
the figures of the accompanying drawings, in which like references indicate similar 
elements, and in which: J 

Figure 1 illustrates a prior art digital decoder for a NTSC/PAL television receiver. 

Figure 2(a) illustrates one prior art implementation of a Y/C separator. 

Figure 2(b) illustrates another prior art implementation of a Y/C separator. 

Figure 3 illustrates one embodiment of a general block diagram of an improved 
video decoder. 

Figure 4 illustrates a block diagram of one general-purpose computer system. 

Figure 5 illustrates a flow chart of one process that one embodiment of improved 
video decoder follows. 



DETAILED DESCRIPTION 



An apparatus and method for decoding a television broadcasting signal are 
disclosed. In the following description, numerous specific details are set forth in 
order to provide a thorough understanding of the present invention. However, it will 
be apparent to one of ordinary skill in the art that the invention may be practiced 
without these particular details. In other instances, well known elements and \ 
theories such as sampling theorem, video processing architecture, video encoding 
techniques, color space conversion, etc. have not been described in special detail in 
order to avoid obscuring the present invention. 

Both National Television Standards Conmiittee (hereinafter NTSC)/Phase 
Alternating Line (hereinafter PAL) composite signals are well-known television 
broadcasting signals and are the sum of a luminance (or brightness) signal and a 
chrominance (or color) signal. Specifically, the luminance signal, or the Y signal, is 
derived from ganmia-corrected red, green, and blue (or RGB) signals. On the other 
hand, the chrominance signal, or the C signal, is derived from the differences 
between the Y signal and the basic red, green, and blue signals. The C signal is 
further modulated onto a color sub-carrier, where the sub-carrier drives one 
modulator at sine phase and drives the other modulator at cosine phase. Under the 
NTSC standard, this color sub-carrier, or a secondary signal that contains additional 
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information to a main signal, operates at approximately 3.58 MHz. Under the PAL 
standards (there are variations), this color sub-carrier operates at either 
approximately 3.58 MHz or 4.43 MHz. 

Additionally, under the NTSC standard, one video frame consists of two 
interleaved fields. Each field contains half of the scan lines in the video frame and is 
displayed in its entirety. Therefore, the odd field is displayed, then the even, then the 
odd, and so on. Moreover, the scan lines from one field are interlaced with the scan 
lines in the next field. A "scan line" refers to an individual sweep across the face of 
the display by the electron beam that makes the video frame. A scan line also 
comprises a collection of pixels. Throughout this disclosure, "digital signal" and 
"digitized signal", are used interchangeably to broadly refer to binary representations 
of a continuous-time, or an analog, signal. The term, "S-video", is used for Super- 
Video, is a technology for transmitting video signals over a cable by dividing the 
video information into two signals, chrominance and luminance signal. Finally, a 
machine readable medium refers to, but not limited to, a storage device, a memory 
device, a carrier wave, etc. 

Figure 3 illustrates one embodiment of a general block diagram of the present 
invention, an improved video decoder (hereinafter IVD). For illustration purposes, 
the following discussions will mainly focus on the NTSC standard in describing IVD 
300. However, it should be apparent to one with ordinary skill in the art to modify 
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the described IVD 300 to operate with other television and video standards, such as 
the various PAL standards. 

One embodiment of IVD 300 processes an analog NTSC/PAL composite 
video signal 312 before the signal reaches display apparatus 324. Analog-to-digital 
converter (hereinafter A/D converter) 302 of IVD 300 samples this video signal 312 
at some sampling frequency to generate digitized video signal 314, which consists of 
a sequence of discrete samples of the continuous-time NTSC/PAL composite video 
signal 312. One implementation of IVD 300 has A/D converter 302 sampling at four 
times of the color sub-carrier frequency, or 4 * 3.58 MHz = 14.32 MHz, under the 
NTSC standard. Further discussions on this particular sampling frequency will be 
provided in the subsequent section that details the operations of IVD 300. 
Additionally, A/D converter 302 is often coupled to a memory device, such as 
memory subsystem 416 or dedicated memory 434 as shown in Figure 4, to store 
samples of digitized video signal 314 for subsequent operations of IVD 300. 

Based on digitized video signal 314, constancy detector 304 then proceeds to 
calculate a constancy value in a number of dimensions, namely, the horizontal, 
vertical and temporal dimensions. The constancy value in the horizontal dimension 
(hereinafter H constancy value) purports to quantify the amount of variation among 
the discrete samples of digitized video signal 314 on a same scan line. The 
constancy value in the vertical dimension (hereinafter V constancy value) focuses on 
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the amount of variation among the discrete samples between different scan lines. 
Lastly, constancy in the temporal dimension (hereinafter T constancy value) mainly 
concentrates on the amount of variation among the discrete samples between frames 
that occur at different points in time. According to comparison resuhs between the 
constancy values and some predefined threshold values, constancy detector 304 
generates selection signal 316 for Y/C separation engine 306. 

One embodiment of Y/C separation engine 306 contains three different 
filters: a horizontal filter, a 2-dimensional (hereinafter 2-D) filter and a 3- 
dimensional (hereinafter 3-D) filter. Based on selection signal 316, Y/C separation 
engine 306 selects an appropriate filter to process digitized video signal 314 and to 
generate filtered signal 318, or filtered Y and C signals. One implementation of a 
horizontal filter involves adding/subtracting discrete samples of digitized video 
signal 314 on a same scan line that are 180^ out-of-phase. One embodiment of a 2-D 
filter involves adding/subtracting discrete samples of digitized signal 314 that are on 
adjacent scan lines, whereas one implementation of a 3-D filter involves 
adding/subtracting discrete samples that are on different frames. Further discussions 
on additional implementation details of the interaction between selection signal 316 
and Y/C separation engine 306 will be provided in the subsequent section that 
details the operations of IVD 300. It should be noted that an ordinary skilled artisan 
may include a different number or different types of filters that have been 
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specifically described in Y/C separation engine 306 without exceeding the scope of 
the present invention. 

Similar to the previously discussed signal adjuster 108 in the Background 
section, signal adjuster 308 operates further on filtered signal 318 to generate 
adjusted signal 320. However, unlike signal adjuster 108, signal adjuster 308 does 
not perform color space conversion in this embodiment. Instead, display encoder 
310 encompasses that functionality. Thus, depending on the type of input format 
display apparatus 324 requires, such as S-video, composite video, etc., display 
encoder 310 ensures that encoded signal 322 is represented in the appropriate color 
space and also conforms to the required input format. 

It should be noted that IVD 300 can be progranmied or impleniented in 
various types of electronic systems. Some examples of such an electronic system 
are, but not limited to, standalone electronic apparatuses, add-on circuit boards and 
general-purpose computer systems. 

A general-purpose computer system 400 is illustrated in Figure 4. The 
general-purpose computer system architecture comprises microprocessor 402 and 
cache memory 406 coupled to each other through processor bus 404. Sample 
computer system 400 also includes high performance system bus 408 and standard 
I/O bus 428. Coupled to high performance system bus 408 are microprocessor 402 
and system controller 410. Additionally, system controller 410 is coupled to 
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memory subsystem 416 through channel 414, is coupled to I/O controller hub 426 
through link 424 and is coupled to graphics controller 420 through interface 422. 
Coupled to graphics controller is video display 418. Coupled to standard I/O bus 
428 are I/O controller hub 426, mass storage 430 and alphanumeric input device or 
other conventional input/output device 432. 

These elements perform their conventional functions well known in the art. 
Moreover, it should have been apparent to one ordinarily skilled in the art that 
computer system 400 could be designed with multiple microprocessors 402 and may 
have more components than that which is shown. Also, mass storage 420 may be 
used to provide permanent storage for the executable instructions of IVD 300 in one 
embodiment, whereas memory subsysteni 416 niay be used to temporarily store the 
executable instructions and samples of digitized video signal 314 during execution 
by microprocessor 402. Alternatively, samples of digitized video signal 314 may be 
stored in.dedicated memory 434 (dotted block shown in Figure 4), which is coupled 
to graphics controller 420. 

Operation of One Embodiment of an Improved Video Decoder 



Figure 5 illustrates a flow chart of one process that one embodiment of IVD 
300 follows. In this embodiment, parts of IVD 300 are represented by a set of 
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executable instructions that are stored in mass storage 420 of general-purpose 
computer system 400 as shown in Figure 4. Additionally, display apparatus 324 as 
shown in Figure 3 correlates to video display 418 in Figure 4 and is also a NTSC 
television with S-video input in one embodiment. 

In conjunction with Figure 3 and Figure 4, this embodiment of IVD 300 
samples NTSC composite signal 312 at 4 times the color sub-carrier frequency in 
block 500. More specifically, this IVD 300* s A/D converter 302 encompasses the 
functionality of one type of input/output device 432 of general-purpose computer 
system 400, which receives and converts the analog NTSC composite signal 312 into 
digitized signal 314. One embodiment of A/D converter 302 also includes some of 
the mentioned executable instructions of IVD 300. Particularly, in block 502, these 
executable instructions cause general-purpose computer system 400 to store an 
appropriate number of samples from digitized signal 314 in either memory 
subsystem 416 or dedicated memory 434 for further processing. 

It is worth noting that NTSC composite signal 312 signal corresponds to a 
combination of a Y signal and a C signal, which is approximately modulated onto a 
3.58 MHz color sub-carrier. Because of the modulation of the C signal, sampling 
NTSC composite signal 312 at (4 * 3.58) MHz results in the first sample of digitized 
signal 314 being 180° out-of-phase with the third sample but in-phase with the fifth 
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sample. The mathematical relationships between some samples of digitized signal 
314 are illustrated with the following equations: 



Equation 1 C = U sin(wt) + Vcos(wt), where w = 2 * pi * sub-carrier frequency 

Equation 2 U * sin(wt + pi) = - U * sin(wt) 

Equation 3 V * cos(wt + pi) = - V * cos (wt) 

Equation 4 Digitized video signal 3 14 = Y + C 

Equation 5 Si = Yi + Ci 

Equation 6 S2 = Y2 + C2 

Equation 7 If Si and S2 are 180° or pi out-of-phase, according to Equations 2 and 
3, 

Si + S2 = (Yi + Ui sin(wt) + Vi cos(wt)) + (Y2 + U2 sin(wt+pi) + V2 

cos(wt+pi)) 

= (Yi + Y2) + (Ui - U2) sin(wt) + (V, - V2) cos(wt) 
Equation 8 If Si and S2 are in-phase, then 

Si - S2 = (Yi + Ui sin(wt) + Vi cos(wt)) - (Y2 + U2 sin(wt) + V2 cos(wt)) 

= (Y, - Y2) + (Ui - U2) sin(wt) + (Vi - V2) cos(wt) 



One embodiment of constancy detector 304 as shown in Figure 3 utilizes the 
mathematical relationships presented above in calculating the H constancy value. 
More particularly, the equation is shown below: 




Equation 9 H constancy value = abs (Si - S2), where "abs" stands for "absolute 
value" and Si and S2 are in-phase. 
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If samples on one scan line (or horizontal samples) do not vary much relative 
to one another, as equation 9 above indicates, the H constancy value of the samples 
would yield a low value. If the H constancy value is less than a predetermined 
threshold value in block 504, constancy detector 304 then informs Y/C separation 
engine 306 to proceed with a horizontal filter to distill the luminance and the 
chrominance information from digitized video signal 314 in block 506. In one 
implementation, constancy detector 304 assumes a low H constancy value for the 
samples that fall between the two in-phase samples if the in-phase samples have a 
low H constancy value. 

As an illustration, assuming that the first sample Si and its in-phase 
counterpart, the fifth sample or S2, do not vary at all, thus Yi = Y2, Ci = C2 and the H 
constancy value = 0. As has been mentioned above, constancy detector 304 assumes 
that the in-between sample, or third sample (S3), remains constant relative to Si. In 
other words, Yi = Y3 and Ci = C3. Because the H constancy value is less than a non- 
zero threshold value in block 504, constancy detector 304 informs Y/C separation 
engine 306 to use a horizontal filter in block 506. Moreover, since S3 is Si's out-of- 
phase counterpart, or 180° out-of-phase, the mathematical relationship in equation 7 
governs the operations of the horizontal filter. Specifically, one embodiment of the 
horizontal filter sums Si and S3 to cancel the chrominance information and to yield 2 
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• Yi (or 2 * Y3). Because digitized video signal 314 = Y + C, this horizontal filter 
can retrieve the chrominance information by subtracting the luminance information, 
which results from the application of equation 7, from digitized video signal 314. 

However, when horizontal samples change rather abruptly, or the H 
constancy value exceeds the predetermined threshold in block 504, one embodiment 
of constancy detector 304 proceeds to determine whether the V constancy value is 
below a predetermined threshold in block 508. More specifically, constancy detector 
304 attempts to detect the amount of changes in a video frame in the vertical 
direction. Under the NTSC standard, it is important to note that the chrominance 
signal's phase is shifted by 180° after each scan line in one field. As an illustration, 
assuming the following: 

• Si represents the first sample of the first scan line 

• S2 represents the first sample of the second scan line 

• S3 represents the first sample of the third scan line 



Equation 10 Si = Yi +Ci 
Equation 11 S2 = Y2 - C2 
Equation 12 S3 = Y3 + C3 
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One embodiment of constancy detector 304 calculates the V constancy value by 
taking the absolute value of the difference between two scan lines that have the same 
chrominance phase. Mathematically, 

Equation 15 V constancy value = abs (Si - S3) 

Similar to the calculation of the H constancy value, this implementation of constancy 
detector 304 assumes that S2 remains constant if S\ does not vary from S3. In other 
words, if Yi = Y3 and Ci = C3, then Yi = Y2 and Ci = C2. In the event that the V 
constancy value is below a predetermined threshold in block 508, constancy detector 
304 informs Y/C separation engine 306 to choose a 2-D filter in block 510. One 
embodiment of the 2-D filter adds samples of successive scan lines, such as Si + S2, 
to cancel the chrominance information and to obtain the luminance information. 

Furthermore, if significant horizontal and vertical variations both exist in a 
field, then constancy detector 304 informs Y/C separation engine 306 to deploy a 3- 
D filter in block 512. Under the NTSC standard, it is important to note that the 
chrominance phase of one field is shifted by 180° in the same field of the successive 
frames. More specifically, 



Si represents the first sample in the first field of the first frame (first in time) 
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• S2 represents the first sample of the first field of the second frame (second in 
time) 

• S3 represents the first sample of the first field of the third frame (third in time) 

Equation 16 Si = Yi +Ci 
Equation 17 S2 = Y2 - C2 
Equation 18 S3 = Y3 + C3 

Again, in order to cancel chrominance information and obtain luminance \ 
information, one embodiment of 3-D filter adds up Si and S2. \ 

Although specific details of one embodiment of constancy detector 304 and 
Y/C separation engine 306 have been described, it should be apparent to one with 
ordinary skill in the art to have significantly different designs or implementations 
and yet still remain within the scope of the present invention. For instance, even 
though portions of Figure 5 illustrate one sequence that one embodiment of 
constancy detector 304 follows, constancy detector 304 is not limited to such 
sequence. For example, constancy detector 304 could start with calculating the T 
constancy value instead of the H constancy value. Altematively, constancy detector 
304 could compute all three constancy values in parallel. 

After Y/C engine 306 separates the chrominance information and the 
luminance information, signal adjuster 308 as shown in Figure 3 further processes 
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filtered signal 318 in block 514. In block 516, display encoder 310 reformats 
adjusted signal 320 to match display apparatus 324's input requirements. This 
process of reformatting includes, but not limited to, color space conversion and pixel 
format conformance, etc. For instance, because display apparatus 324 is assumed to 
have S-video inputs, one embodiment of display encoder 310 relays adjusted signal 
320 to display apparatus 324 without performing any color space conversion. 
Display encoder 310 nevertheless needs to make certain that the pixel data from IVD 
300 are in recognizable formats (such as 24-bit RGB, 16-bit 4:2:2YcbCr, etc.) to 
display apparatus 324. On the other hand, if display apparatus 324 requires RGB 
inputs, display encoder 3 10 first converts adjusted signal 320, which is in Y/C color 
space, into RGB color space and then encodes the converted signal into a RGB pixel 
format that meets the input requirements of display apparatus 324. 

It is important to note that functionality of the disclosed blocks in Figure 3 
can either be combined or further divided without exceeding scope of the present 
invention. As an example, signal adjuster 308 and display encoder 310 can be 
combined into one general block. On the other hand, signal adjuster 308 can be 
further divided into blocks such as a demodulator block, display enhancement 
processing block, etc. 

Thus, an apparatus and method for decoding a television broadcasting signal 
have been described. Although an improved video decoder has been described 
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particularly with reference to the figures, it will be apparent to one of the ordinary 
skill in the art that the present invention may appear in any number of other 
electronic systems. It is contemplated that many changes and modifications may be 
made by one of ordinary skill in the art without departing from the spirit and scope 
of the present invention. 



